Winning Faces: Model of basic primate visual processing predicts elections

Todorov (2005) demonstrated that people could make competence judgments about faces with only 100 millisecond exposures that were predictive of electoral victories. Others have since shown that Swiss children can predict the outcome of French elections when they pick which face they would want to be the 'captain of the boat' in an adventure game. And, Indians and Americans can pick the winners of Mexican and Brazilian political contests just by looking at their faces. The automaticity, early development, and universality of this ability suggests it is likely instantiated in a basic cognitive process. We trained a computer model of the V1 region of the primate visual cortex that has performed very well in object recognition tasks to identify winning candidates with an accuracy of 70-80%. Voters must rely on mental shortcuts (such as party affiliation, ideology, endorsements, success in polls, and appearance) rather than on a complete knowledge of political candidates’ policy preferences, qualifications, and alliances when choosing a representative (1). Despite very low levels of political knowledge and the use of these heuristics, voters are able to correctly choose a candidate that matches their policy preferences 75% of the time (2). As primates, our ability to quickly navigate the complex and dynamic politics of our social context is critical to survival and we need to make nearly instant judgments about the likely winners and losers in tribal conflicts (3). Our modern politics appear to be constructed on these very ancient foundations (4). Attributions about personal characteristics of political candidates made solely on the basis of facial appearance have been repeatedly demonstrated to predict election results (5). Since these judgments can be made based on only 100 millisecond glimpses, can be made by young children (6), and appear to be culture independent (7), we hypothesized that these assessments might be the result of rapid, automatic visual processes. Although other brain structures are known to be involved in complex social cognition tasks, the primate’s ventral visual processing stream is critical for swiftly discriminating among visually-presented objects (8), including faces. To test whether basic ventral visual processes could be sufficient for predicting electoral outcomes from only facial appearance, we began by using a computer algorithm that models only the first stage of primate cortical visual processing – the primary visual cortex (area V1). This “V1-like” model functions as a neuroscience “null” model because it is only a first order description of the way the early visual system codes visual images and would not a priori be expected it to perform well with real-world object and face recognition tasks. Nonetheless, in some contexts the output of this relatively simple visual code can beat other state of the art computer vision approaches (9). We trained the standard linear classifiers on the V1-like visual code, with some of the same set of candidate photos that have been used in many studies with human subjects (5) and then tested the election prediction performance using photos of candidates that it had not previously seen. Using only face-cropped photos, we found that this V1-like model predicted previous U.S. Senate and House of Representatives races with accuracy comparable to human subjects in previous appearance-based studies (U.S. House 74.6% correct, sem: 2.3%; U.S. Senate 75.5% correct, sem: 2.4%; chance is 50%). Because performance was much lower using only a raw pixel code (55.1% and 51.4% respectively), this suggests that the predictive power results from the way the primate brain represents those images (approximated here by the V1-like model). To test generalizability across cultures, we first used the same V1-like model (without any additional training data) to successfully predict elections in Mexico (55.6% correct) and Brazil (70.4% correct). Second, we made accurate (65.1% correct, p<0.001) prospective predictions of the 2010 U.S. House, Senate, and Gubernatorial elections using the V1-like model trained on previous elections. Consistent with studies using human raters (10), the accuracy depended on whether the election was in an uncompetitive (75.8% correct, p<0.001) or highly competitive district (35.1% correct, p=0.16). In districts that are evaluated to be more politically competitive, both parties appear to put their “best face forward” neutralizing any facial appearance advantage. The algorithm appears to rely on facial features similar to those that are primarily attended to by humans (e.g. eyes, nose, mouth) when engaged in face processing (Figure 1). While it remains unclear precisely what characteristics of the image are enabling this algorithm to make accurate forecasts of electoral results and it is unclear if humans rely on these same characteristics, this computational approach should enable a more systematic analysis of the relationship between appearance and electoral outcomes. Although many candidate characteristics that drive voting behavior can be readily identified (party affiliation, endorsements) or quantified (ideology, polling results) using traditional tools of political science, candidate appearance has been far more difficult to study rigorously. These results suggest that features of a candidate’s appearance that are identifiable by rapidly computed, low-level visual processes are predictive of electoral outcomes. Since the complexity of human politics demands that we rely upon shortcuts to make decisions (1), further study of the neural mechanisms which implement these shortcuts is critical for our understanding of the democratic process. Fig. 1. Red areas show regions of the face where differences in the two candidates’ appearance had the strongest impact on predictions of electoral outcome. These regions and the detailed weighting on the underlying V1-like visual feature (see methods) were discovered from photos of pairs of political candidates in which the electoral outcome of each race was provided. For reference, an example image of a candidate is shown in the background.